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Image Capture Systems using Motion Detection 



This invention relates to an image capture system that 
uses at least one motion sensor. 

It is known for a user to wear a head mounted camera, 
which points in the direction that the user's face is 
facing, so that images can be captured of roughly what the 
user is looking at . 

This type of set-up does not allow for the user moving his 
eyes relative to his head, causing resultant images to 
often be poorly framed. 

Head-mounted cameras are well known, such as that 
described in W0994 9656 (Mann) . The system described in 
Mann assumes that the head direction is the correct 
direction for image capture. Any control of what is 
captured needs to be done by consciously pointing the head 
in the direction of interest. Sophisticated wearable 
camera systems such as described as Mann, are coupled with 
a wearable view finder to provide the wearer with 
confirmation that what the camera is seeing is what is 
required. 

In Wearable Visual Robots (IEEE International Symposium on 
Wearable Computing, ISWC'OO, Atlanta, October 2000) a 
wearable visual robot is disclosed having a pan/tilt 
mechanism. The work described mounts the apparatus on a 
wearer's shoulder. The device has motion sensors attached 
to it, but there are no sensors attached to the wearer's 
head, so the device is unable to take into account the 
motion of the wearer's head. The aim of the approach in 
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this document is to provide a sensor which "decouples 
camera movement from the wearer's posture and movements". 
The aim is to allow the robot itself to choose what to 
look at. The motion sensors are provided to increase its 
5 ability to stabilise its line of vision on what the robot 
decides should be captured, regardless of where the user' s 
attention is focussed. 

Several alternatives have also been proposed for capturing 
10 a panoramic image of a scene. These include US 614034 
which discloses a dodecahedral arrangement of sensors 
whose images can be combined into a single panoramic 
image. Also, US 6356296 describes the use of a parabolic 
reflector to capture an entire 180° angle of view using a 
15 single sensor. The aim of both of these systems is to 
-provide a very wide angle of view image. A problem arises 
with panoramic view capture in that it is not clear what 
part of the image is of interest. Typically a panoramic 
view will not all be interesting, but merely a part 
2 0 thereof. 

It is an object of the present invention to address the 
above mentioned disadvantages. 

25 According to a first aspect of the invention an image 
capture system incorporates an image capture device and 
head motion detection means operable to detect motion of a 
user's head; wherein the image capture system is adapted 
to move a direction of a field of view of the. image 

30 capture device relative to the user's head, based on 
detected motion of the user's head. 
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The image capture device is preferably operable to be 
mounted on a user's head. 

The head motion detection means are preferably adapted to 
5 be secured to a user's head. 

The image capture system may be adapted to move, 
preferably rotate, the field of view of the image capture 
device relative to a direction of view of the user's face. 

10 

The head motion detection means may be mechanical head 
motion detection means, preferably at least one gyroscope 
and/or at least one compass. 

15 The head motion detection means may be image analysis 
means operable to determine a motion of an object within a 
field of view of the image capture device to infer motion 
of a user's head. 

2 0 The head motion detection means may incorporate a 
centering mechanism, which preferably takes as a centre 
position a position adopted by the motion detection means 
for a majority of the time. 

2 5 The image capture system is preferably operable to control 

a direction of view of the image capture device with field 
of view control means, preferably to control a field of 
view by a lateral rotation. 

3 0 The field of view control means may control at least one 

additional image capture device, preferably a lateral 
additional image capture device, preferably two lateral 
additional image capture devices. A lateral image capture 
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device may be an additional image capture device, 
preferably positioned such that an optical axis thereof is 
rotated in a horizontal plane relative to that of the 
first mentioned image capture device, preferably thereby 
5 having a field of view which is horizontally rotated from 
the field of view of a main image capture device. 

The or each lateral additional image capture device 
preferably has a field of view to one side of a main image 
10 capture device that substantially partially overlaps with 
a field of view the main image capture device. 

Images from the main and additional image capture devices 
may be buffered in a storage means for subsequent 
15 retention of an image/ images from one of said image 
capture devices . 

The field of view control means may comprise a mechanism 
operable to adjust a direction of the image capture 
20 device, preferably to adjust the direction from side to 
side. The mechanism may be a pan adjustment mechanism, or 
may be a pan/tilt adjustment mechanism. 

The field of view control means may comprise at least one 
25 mirror operable to be adjusted to adjust an image 
reflected to the image capture device. Preferably, the 
field of view control means comprises two mirrors. The 
field of view control means may be operable to adjust the 
or each mirror to move the image reflected to the image 
30 capture device laterally. 

The field of view control means may be operable to select 
a smaller image from a first image captured by the image 
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capture device. Said first image may be an image of 
substantially all of a user's lateral field of view, 
including the user' s view with his eyes moved to one or 
both sides. The image capture device may incorporate a 
5 wide-angle lens, which may be a fish-eye lens. 

The image capture system may incorporate control means, 
which may be a computer or microcomputer, operable to 
activate the field of view control means on detection of 

10 movement of the user's head. Preferably, on detection of 
movement of the user's head in a first direction by the 
head motion detection means, the control means is operable 
to activate the field of view control means to move a 
direction of view of the image capture device in the first 

15 direction, by an amount in addition to the detected head 
motion. 

An amplitude of movement of the user' s head detected by 
the head motion detection means may be used to control an 
20 amplitude of movement of the direction of view of the 
image capture device in addition to the detected head 
motion. The amplitude of movement of the field of view 
may be chosen to be of substantially the same amplitude as 
the amplitude of detected head movement, 

25 

A direction of detected head movement may be used to 
select an additional or lateral image capture device for 
providing the field of view. 

3 0 According to a second aspect of the invention a method of 
capturing an image with an image capture device, 
comprises : 

detecting motion of user's head; and 
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offsetting a field of view of an image capture device 
in a direction of the detected motion of the user's head. 

The image capture device may be secured to the user' s 
5 head. 

Preferably, the detected motion is a rotation of the 
user's head, preferably a lateral rotation, to left or 
right . 

10 

The field of view may be offset by an amount derived from 
an amplitude of the detected motion, preferably in the 
range of approximately 30° to approximately 45°. 

15 All of the features described herein may be combined with 
any of the above aspects, in any combination. 

For a better understanding of the invention and to show 
how the same may be brought into effect, specific 
2 0 embodiments will now be described, by way of example, with 
reference to the accompanying drawings, in which: 

Figure 1 is a schematic front view of an image capture 
device incorporating three separate image sensors pointing 
25 at different angles and incorporating a motion sensor, for 
wear as a pair of spectacles; 

Figure lb is a schematic view from above of the device 
shown in Figure la; r 

30 

Figure 2a is a schematic view from the front of an image 
capture device having a pan/tilt mechanism for adjusting 
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sensor ; 

Figure 2b is a schematic view from above of the device 
5 than in Figure 2a; and 

Figure 3 is a partial schematic view of an alternative 
arrangement of image capture device having a switchable 
direction of view by means of pivotable mirrors; 

10 

Figure 4a is a schematic view from above of a camera 
having a fish -eye lens; and 

Figure 4b is a schematic view and field of view selection 
15 from a larger image . 

It has been realised that when a person turns his head to 
look at a new stimulus his eyes are typically moved more 
quickly than his head so that as he looks to the side his 

20 eye movement, combined with the head movement, rotates his 
direction of view by up to twice the angle of the head 
rotation alone. If a camera is fixed to a user's head 
then it will only rotate by the head movement, which 
results in the camera no longer capturing approximately 

25 what the wearer is looking at. Below are described 
various embodiments by which an image sensing means can 
offset the direction of view of a captured image from the 
direction of a user's head, such that the direction of the 
image capture rotates in the same direction as the head, 

3 0 but by a larger angle. This accomplished by sensing the 
rotation of the user's head, detecting when a turn to the 
left or right is occurring and moving the angle of view of 
the image sensing means so that it rotates further than 
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the rotation of the head alone, in the same direction as 
the head rotation. 

The embodiments described disclose a system that adjusts 
the direction of image capture to take into account eye 
motion, rather than simply relying on head direction. 
This frees a user from having to think about what is being 
captured, because the wearable camera system - will 
generally be capturing the area of visual attention more 
accurately than a conventional head mounted camera. 



The embodiments described below all require the head 
rotation detection mechanism to control an additional 
change in the direction of view of the captured image 
15 beyond mere movement of the user's head. Also, an image 
capture device having a controllable field of view is 
required. 

In relation to head rotation detection a first head 
20 rotation detection mechanism 10 is shown in Figures la and 
lb mounted on a pair of spectacles 12 to be worn by a user 
14. The head rotation detection mechanism 10 makes use of 
a mechanical motion sensor. 

25 This may be in the form of a micro-compass which gives an 
absolute measure of the direction in which the detection 
mechanism 10 is pointing. Suitable calibration to a 
forward direction can be used to provide an angle of 
deviation from forward to indicate how much a user's head 

3 0 has moved. A suitable type of compass would be a Hall 
effect type, which may have a low accuracy as low as 45°, 
but may be sufficient to indicate that a person's head has 
moved to either right or left. Such a low accuracy device 
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has the benefit of low cost. A more accurate compass 
would be a Honeywell HMR 3200 model, which gives a half 
degree accuracy. 

5 An alternative type of head rotation detection mechanism 
device 10 would be to use a miniature gyroscopic device, 
such as a Murata Enc 03J, which is of a piezoelectric 
type, in which a vibrating column is moved off -axis by 
acceleration of the device, which movement off -axis causes 
10 current to flow in a piezoelectric device. With a 
gyroscope, the rate of movement need only be integrated to 
revert to an angle through which the accelerator has 
-rotated. The amount of rotation is determined by 

integration of the signal. 

15 

The output of an accelerometer type gyroscope will provide 
a peak in a first, e.g. positive, direction for a leftward 
rotation of the user's head and a second, e.g. negative, 
peak for opposite, rightward direction of the user's head. 

20 These would provide positive and negative values for an 
integral of the signal. Thus, left and right motion can 
easily be determined by a positive value for leftward 
movement and a negative value for rightwards movement. The 
amount of the integral provides the amplitude of the 

25 rotation in that given direction. 

Normally, in use, following a rotation of his head, a user 
14 will return his head to the central position with his 
head pointing forward. In addition, a user will typically 
30 have his head pointing forwards for the majority of the 
time. Both of - these facts can be used to provide a 
resetting mechanism for the head rotation detection 
mechanism 10 which can be set to have the forward 
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direction as the direction used for the majority of the 
time . 

An alternative method by which head rotation can be 
5 detected would be to use the analysis of high frame rate 
imagery from a camera view in line with the user's head in 
order to detect head rotation from the motion field of the 
video. Taking a camera 16 as shown in Figure la, which 
captures images at a high frame rate, movement of the 

10 image is detected. For example images may be coded with an 
MPEG standard, in which a difference between successive 
images is calculated and coded, rather than coding the 
whole image for each " frame" . By taking a crude average 
of the position of selected blocks within an image and 

15 analysing how those blocks move across the screen, an 
estimate of head rotation speed can be obtained. More 
specifically, corner points or edges of objects within the 
field of view can be detected and located and then tracked 
across the field of view to give an estimate of an amount 

20 rotation of a user's head. This latter method would need 
a reasonable degree of focus in order to be able to detect 
corner points. Further information can be obtained from 
Comparing and Evaluating Interest Points; Cornelia 
Schmid, Roger Mohr, and Christian Bauckhage; INRIA Rhone- 

25 Alpes, 655 ar d'L Europe, 28330, Montbonnot , France. 

Together with detection of head mot ion/ rot at ion, it is 
also necessary to have a controllable field of view for 
image capture. A number of alternatives are possible. 

30 

The first alternative is to use multiple cameras angled in 
fixed orientation such that a captured image can be 
switched automatically from one to other camera, with the 
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possibility of forming a view from an intermediate angle 
of view by forming a composite from parts of an adjacent 
pair of cameras. Figure la and Figure lb show a central 
camera 16, a left camera 18, and a right camera 20 having 
5 the fields of view shown schematically in Figure lb. The 
fields of view clearly overlap. 

The embodiment shown in Figures la and lb would typically 
be used by recording the central camera 16 when a wearer's 

10 head is not rotating, with the other cameras 18 and 2 0 
being ignored. When rotation of the user's head to the 
left for example is detected by the motion detection 
device 10 recording the image is switched to camera 18 . 
When the user's head returns to the centre position, as 

15 detected by the motion detector sensor 10, recording is 
switched to the central camera 16. Similarly, when a 
right turn of the head is detected recording is switched 
to the right camera 2 0 and back to the central camera 16 
when the user's head moves back to the centre position. 

20 

The pair of spectacles 12 shown in Figures la and lb 
carries the three cameras 16, 18 and 2 0 and the motion 
detection device 10. The motion detection device may be 
located on an arm 13 of the spectacles to be less 
25 obtrusive. Also, a control and image recording apparatus 
22a may be located on the glasses. Alternatively, images 
may be transmitted to a remote control and image storage 
section 22b, for example by a wireless link, such as a 
Blue Tooth link, or by a wired link. 

30 

Typically, the spectacles 12 incorporating the cameras 16, 
18, 2 0 and motion detection means 10 will continuously 
capture images from one of the cameras 16, 18, 20. Frames 
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may be deliberately dropped when there is excessive head 
motion if the exposure time cannot be kept sufficiently 
high to avoid motion blur. The frames may be stored 
continuously as a video, or they may be held in a 
temporary rolling buffer such that a predetermined number 
of the last images is recorded. In this case, the user 
would typically signal to the camera when something that 
should be recorded had just been observed. On receiving 
the signal the control /storage apparatus 22a or 22b would 
transfer the current contents of the rolling buffer to its 
permanent storage, also located in the control /storage 
section 22a/22b. 

Normally, following a rotation of the head, the camera 
wearer will return his head to the central position within 
a short time. This can be detected by the head rotation 
sensor 10 to cause the image capture to revert to the 
centrally positioned camera 16. If a return rotation is 
not detected within a short time, typically in 
approximately 2 seconds, the controller/ storage apparatus 
22a/b automatically returns to recording of the central 
view from camera 16, because it is unlikely that the 
wearer will keep his eyes far from straight for a 
prolonged time. Instead, the body is likely to be turned 
towards the direction of interest. 

This automatic return to centre may be implemented by a 
simple time-out mechanism. Alternatively, the system may 
revert to centre when it has a high confidence that an 
image of acceptable quality has been captured, looking in 
an extreme direction. Such a confidence measure can be a 
function of camera motion, exposure and focus parameters 
for example, by which it may be determined whether the 
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camera has been stationary for long enough for a suitable 
image to be captured. 

The multiple sensor implementation shown in Figures la and 
lb has the advantage that images from the side views can 
be captured immediately. This is typically important 
because it is likely to have been an interesting event 
which the wearer saw "out of the . corner of his eye" which 
attracted his attention and hence gave rise to head 
rotation. A camera that is already pointing in the 
correct direction has a high chance of capturing an image 
of the event . 

Still further, the latter advantage may be further 
improved by implementing the system so that each of the 
three cameras 16 to 20 in Figures la/b is locally 
buffering images to the storage/control section 22a/b, 
with only one camera output being retained. When inputs 
to the system of the motion sensors to control section 
2 2a/b indicate that images should be captured the buffered 
images could be taken from a time fractionally earlier 
than the detection time. In this way, the view that 
caused the user to turn his head will be caught before he 
actually turned. 

The advantages of the embodiment shown in Figures la/b 
could be improved by using sufficient cameras with 
overlapping fields of view that at least one of them will 
always capture an image which has reasonable framing. If 
an insufficient number of cameras is used, then framing 
may not be adequate. Also, a sufficiently fast exposure 
should be used such that images free from motion blur are 
captured even whilst the user's head is rotating. In this 
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way, an acceptably cropped still image will be obtained by 
at least one of the frames captured during the head 
rotation. Alternatively by using the three cameras shown 
in Figures la and lb frames may be combined from adjacent 
5 cameras by using any of the well-known mosaicing 
techniques in order to generate an image with an 
intermediate direction of view, but which is better 
framed . 

10 As an alternative to the three cameras 16 to 18 shown in 
Figures la/lb, it is possible to arrange a pan/tilt 
mechanism 2 4 for a camera 2 6 to be mounted the user's 
head, as shown in Figures 2a/b. A pan/ tilt mechanism 
similar to the one described in Wearable Visual Robot 

15 (above) could be used. 

The pan/tilt mechanism 24 is controlled in the same way 
that camera selection is made in the previous example, 
except that when a side view (left or right) is required 

20 the pan/tilt mechanism 24 moves the camera 26 to the side 
by a given angle. The angle use may be approximately in 
the range of approximately 30° to approximately 4 5° , which 
corresponds to an angle through which a user's eyes may 
typically move when glancing sideways to follow a 

25 movement. 

A further alternative would be to use a system shown in 
Figure 3 by which first and second fixed sensors 2 8 and 32 
have an optically rotatable or switchable direction of 
30 view. This system is effectively like a periscope on its 
side, with a moveable mirrors 30 that can be rotated (as 
shown by the arrows) in order to change the image that is 
reflected towards the sensors 2 8/32 from a view V. The 
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two sensors 28/32 may face towards a centre part of the 
spectacles shown in Figure 4a. The mirror 30 (or a mirror 
for each sensor) may be rotated to change the field of 
view reflected to the sensors 28/32. Mirror 28 may 
5 reflect for a right hand field of view and the other 
mirror (not shown) for a left hand field of view. An 
alternative arrangement would use a single mirror. 

For the embodiment described above, the arrangements 
10 described in Figures 2a, 2b and 3 have the advantage that 
the number of components is reduced, because fewer image 
sensors are required. However, there may be some time lag 
disadvantage with these embodiments, because of the time 
required to sense the need for movement to one side and 
15 the time to achieve that movement. 

A further alternative for controlling the field of view of 
the image capture device, would be to have automated 
cropping of an image from a camera 4 0 with a very wide 

20 field of view 42 (see Figure 4a) . This would need a high 
resolution sensor 44 and would also typically use a fish- 
eye lens 46 or the like, A window of interest 48, as 
detected and defined by a user's head motion in the manner 
described above could then be moved across the captured 

25 field of image 42 to retain that part of the image which 
is decided to be of interest. 

An advantage of this latter controllable field of view is 
that there are no moving parts. In the function required 
3 0 by the system described herein good image quality is 
required at the periphery of an image also, because that 
may become the centre of a chosen field of interest 4 8 
within the image 42 . 
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A more sophisticated implementation of the systems 
described above would be to include an additional 
mechanism for detecting rotation of the user's body. Such 
a mechanism would allow head motion relative to the body 
to be identified- The mechanism used would be the same as 
that used to detect head rotation, but merely secured to 
the user's body instead of his head. This would allow the 
head motion relative to the body to be identified. With 
this embodiment an offset view angle can be chosen on the 
basis of the measured head rotation relative to the 
measured body rotation. This embodiment has the advantage 
of being a more accurate indicator of when the additional 
rotation of view should' be started and terminated, because 
it may be used to prevent inappropriate changes to the 
camera field of view. For example if a wearer is walking 
around a corner, he may look straight ahead so no change 
of the field of view is required. The addition of a body 
rotation sensor allows this situation to be differentiated 
from that of a head turn alone, the latter signifying a 
change of view. . 

Another use of this embodiment is in the case where the 
head turns initially relative to the body, but the body 
then turns by a matching amount, so that head and body are 
facing the same way. The time at which the body "catches 
up" can signify that the focus of attention has reverted 
to straight ahead, so the additional rotation of the 
camera's field of view can be terminated when this 
condition is detected, if it has not been detected 
earlier. 
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It would be optional for any of the methods for capturing 
images described above to be left as a final choice to a 
user, or a best frame technique could be used to 
automatically pick the angle of view which best frames a 
5 probable sub j ect . 

All of the embodiments described above advantageously 
provide means by which a user can automatically capture an 
image of a subject of interest to which his attention is 
10 drawn and to which he is moving his eyes relative to 
movement of his head. 
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CLAIMS : 
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1. An image capture system incorporates an image capture 
device and head motion detection means operable to detect 
motion of a user's head, wherein the image capture system 
is adapted to move a direction of a field of view of the 
image capture device relative to the user's head, based on 
detected motion of the user's head. 

2. An image capture system as claimed in claim 1, in 
which the image capture device is operable to be mounted 
on a user's head. 

3. An image capture system as claimed in either claim 1 
15 or claim 2, in which the head motion detection means are 

adapted to be . secured to the user's head. 

4. An image capture system as claimed in any preceding 
claim, in which the head motion detection means are 
operable to detect rotation of the user's head. 
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5. An image capture system as claimed in any preceding 
claim, in which the head motion detection means are 
mechanical head motion detection means. 

6. An image capture system as claimed in any preceding 
claim, in which the head motion detection means 
incorporate a centering mechanism. 

7. An image capture system as claimed in any preceding 
claim, in which the image capture system is operable to 
control the direction of view of the image capture device 
with field of view control means. 
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8. An image capture system as claimed in claim 7, in 
which the field of view control means comprise at least 
one additional image capture device. 

9. An image capture system as claimed in claim 8, in 
which the or each additional image capture device is a 
lateral image capture device, which has a field of view to 
one side of a main image capture device that substantially 
partially overlaps with a field of view of the main image 
capture device. 

10. An image capture system as claimed in claim 8 or claim 
9, in which images from the main and additional image 
capture devices are buffered in a storage means for 
subsequent retention of an image/ images from one of said , 
image capture devices. 

11. An image capture system as claimed in claim 7, in 
which the field of view control means comprise a mechanism 
operable to adjust a direction of the image capture 
device. 

12. An image capture system as claimed in claim 7, in 
which the field of view control means comprise at least 
one mirror operable to be adjusted to adjust an image 
reflected to the image capture device. 

13. An image capture system as claimed in claim 7, in 
which the field of view control means are operable to 
select a smaller image from a first image captured by the 
image capture device. 
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14. An image capture system as claimed in any one of 
claims 7 to 13, in which the image capture system 
incorporates control means, operable to activate the field 
of view control means on detection of movement of the 

5 user's head, 

15. An image capture system as claimed in claim 14, in 
which, on detection of movement of the user's head in a 
first direction by the head motion detection means, the 

10 control means is operable to activate the field of view 
control means to move a direction of view of the image 
capture device in the first direction, by an amount in 
addition to the detected head motion. 

15 16. An image capture system, as claimed in any preceding 
claim, in which an amplitude of movement of the user's 
head detected by the head motion detection means is used 
to control an amplitude of movement of the direction of 
view of the image capture device in addition to the 

20 detected head motion. 

17. An image capture system as claimed in any preceding 
claim, in which a direction of detected head movement is 
used to select an additional image capture device for 

25 providing the field of view. 

18. A method of capturing an image from an image capture 
device , comprises : 

detecting motion of a user's head; and 
30 offsetting a field of view of an image capture device 

in a direction of the detected motion of the user's head. 
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19. A method as claimed in claim 18, in which the image 
capture device is secured to the user's head. 

20. A method as claimed in either claim 18 or claim 19, in 
5 which the detected motion is a rotation of the user's 

head. 

21. A method as claimed in any one of claims 18 to 20, in 
which the field of view is offset by an amount derived 

10 from an amplitude of the detected motion. 

22. An image capture system substantially as described 
herein with reference to the accompanying drawings. 

15 23 . An method of capturing an image from an image capture 
device substantially as described herein with reference to 
the accompany drawings. 
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Abstract 

Image Capture Systems using Motion Detection 

An image capture system incorporates an image capture 
5 device (16) and head motion detection means (10) operable 
to detect motion of a user' s head, wherein the image 
capture system is operable to move a field of view of the 
image capture device (16) based on a detected head motion. 
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